Decode `meta_data` incorrectly sent as an object #16141

joshheald · 2025-09-17T17:06:30Z

Description

Some plugins can break the expected JSON for meta_data on Products and Variations. They change the expected array to an object, with string indexes as keys.

{
  "0": {
    "id": 1794,
    "key": "_wpcom_is_markdown",
    "value": "1"
  },
  "1": {
    "id": 46321,
    "key": "_yoast_wpseo_primary_product_cat",
    "value": ""
  },
}

But it should be an array:

[
  {
    "id": 1794,
    "key": "_wpcom_is_markdown",
    "value": "1"
  },
  {
    "id": 46321,
    "key": "_yoast_wpseo_primary_product_cat",
    "value": ""
  },
]

This PR updates our decoding to support this format. Note that we do not re-encode to the incorrect format, so saving custom fields may be broken in this place, but at least they'll be visible.

Steps to reproduce

Ensure you have a product with non-private custom fields (no leading underscore in the key.)

Apply the snippet below to your site to force the incorrect format
Launch the app and open a product with custom fields set
Observe that you can see the custom field data


/**
 * Transform WooCommerce product & variation meta_data
 * from array to object keyed by array index in REST API responses.
 */
function wc_meta_data_array_to_object( $response, $object, $request ) {
    if ( ! ( $response instanceof WP_REST_Response ) ) {
        return $response;
    }

    $data = $response->get_data();

    if ( isset( $data['meta_data'] ) && is_array( $data['meta_data'] ) ) {
        $transformed = [];
        foreach ( $data['meta_data'] as $index => $item ) {
            $transformed[ (string) $index ] = $item;
        }
        $data['meta_data'] = (object) $transformed;
        $response->set_data( $data );
    }

    return $response;
}

add_filter( 'woocommerce_rest_prepare_product_object', 'wc_meta_data_array_to_object', 20, 3 );
add_filter( 'woocommerce_rest_prepare_product_variation_object', 'wc_meta_data_array_to_object', 20, 3 );
add_filter( 'woocommerce_rest_prepare_shop_order_object', 'wc_meta_data_array_to_object', 20, 3 );

Test information

I've checked viewing Custom Fields on Orders and Products, and Interac refunds (which rely on _charge_id.)

Subscription products also use it, but for the moment I've only used unit tests to verify those.

I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

wpmobilebot · 2025-09-17T17:16:15Z

📲 You can test the changes from this Pull Request in WooCommerce iOS Prototype by scanning the QR code below to install the corresponding build.

	App Name	WooCommerce iOS Prototype
	Build Number	`pr16141-fa134fa`
	Version	`23.2`
	Bundle ID	`com.automattic.alpha.woocommerce`
	Commit	`fa134fa`
	Installation URL	3h14gbctu82ko

Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

joshheald · 2025-09-17T17:48:49Z

@staskus not urgent, just when you get to it.

Copilot

Pull Request Overview

This PR implements a workaround to handle incorrectly formatted meta_data fields in WooCommerce REST API responses. Some plugins transform the expected array format into an object with string indices as keys, breaking the standard JSON structure for Products, Variations, and Orders.

Adds flexible decoding that supports both array and dictionary formats for metadata
Updates Product, ProductVariation, and Order models to use the new flexible decoder
Maintains backward compatibility while fixing visibility of custom fields

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
RELEASE-NOTES.txt	Documents the workaround for incorrectly formatted meta_data
MetaDataArray+FlexibleDecoding.swift	New extension providing flexible decoding for MetaData arrays
ProductMetadataExtractor.swift	Updated to handle both array and dictionary metadata formats
Product.swift	Modified to use flexible metadata decoding
ProductVariation.swift	Modified to use flexible metadata decoding
Order.swift	Modified to use flexible metadata decoding
Test files	Added comprehensive test coverage for both metadata formats

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-18T06:41:28Z

Modules/Sources/NetworkingCore/Model/MetaDataArray+FlexibleDecoding.swift

+
+        // Try to decode as object keyed by index strings
+        if let metaDataDict = try? container.decode([String: MetaData].self, forKey: key) {
+            return Array(metaDataDict.values)


Using Array(metaDataDict.values) loses the ordering information from the dictionary keys. Since the keys are string indices ('0', '1', '2'), the metadata should be sorted by these keys to preserve the original order. Consider sorting by converting keys to integers: return metaDataDict.sorted { Int($0.key) ?? 0 < Int($1.key) ?? 0 }.map { $0.value }

Suggested change

return Array(metaDataDict.values)

return metaDataDict.sorted { (lhs, rhs) in

(Int(lhs.key) ?? 0) < (Int(rhs.key) ?? 0)

}.map { $0.value }

I wonder if it's true, the docs suggest:

When iterated over, values appear in this collection in the same order as they occur in the dictionary’s key-value pairs.

Copilot · 2025-09-18T06:41:29Z

Modules/Sources/Networking/Model/Product/ProductMetadataExtractor.swift

-            newDict.updateValue(objectValue, forKey: objectKey)
-            return newDict
+    private func getKeyValueDictionary(from metadata: [MetaData]) -> AnyDictionary {
+        metadata.reduce(into: AnyDictionary()) { dict, object in


[nitpick] The reduce(into:) method is more efficient than the previous reduce() approach since it mutates the accumulator in-place rather than creating new dictionaries on each iteration.

staskus

Thanks! Solid work, I appreciate the clear test coverage.

I think it can be clean up slightly by removing the Decodable path from ProductMetadataExtractor.

Modules/Tests/NetworkingTests/Mapper/OrderMapperTests.swift

staskus · 2025-09-18T06:30:22Z

Modules/Sources/NetworkingCore/Model/MetaDataArray+FlexibleDecoding.swift

+        }
+
+        // Fallback to empty array
+        return []


Maybe it would be worth logging a case where metadata decoding fails silently? It could help troubleshoot similar issues more easily.

staskus · 2025-09-18T06:40:09Z

Modules/Sources/NetworkingCore/Model/MetaDataArray+FlexibleDecoding.swift

+        }
+
+        // Try to decode as object keyed by index strings
+        if let metaDataDict = try? container.decode([String: MetaData].self, forKey: key) {


I was checking if a plugin could produce [Int: MetaData].self, but it looks unlikely since using numbers as object keys goes against the JSON specs and json_encode() would ensure that the keys are strings.

Yeah. I'm wary of even doing this much workaround for plugins messing with fields that don't belong to them, to be honest... it seems like a short road to insanity trying to pair that with our type safe models.

It'll continue to fail gracefully even if the format is more broken than this.

staskus · 2025-09-18T06:48:09Z

Modules/Sources/NetworkingCore/Model/MetaDataArray+FlexibleDecoding.swift

+
+        // Try to decode as object keyed by index strings
+        if let metaDataDict = try? container.decode([String: MetaData].self, forKey: key) {
+            return Array(metaDataDict.values)


I wonder if it's true, the docs suggest:

When iterated over, values appear in this collection in the same order as they occur in the dictionary’s key-value pairs.

staskus · 2025-09-18T06:56:50Z

Modules/Sources/Networking/Model/Product/ProductMetadataExtractor.swift

        let container = try decoder.singleValueContainer()
-        self.metadata = try container.decode([DecodableDictionary].self)
+
+        // Try to decode as array first (standard format)


I was investigating why we need this duplicating logic, and I think the whole init(from decoder: Decoder) throws and Decodable from ProductMetadataExtractor can be removed. I couldn't find any code paths that rely on it. ProductMetadataExtractor is used only through init(metadata:) initializer.

Thank you, I'll check that.

However, I tried a lot last night to have a sane, non-duplicative approach, and failed hard. It was down to the subtle differences of what data source we were trying to map, and how decoded it already was.

Hopefully the cold light of day will let me delete more of this.

Co-authored-by: Povilas Staskus <[email protected]>

joshheald · 2025-09-18T09:13:56Z

I wonder if it's true, the docs suggest

Thanks for the docs. Either way, it doesn't matter – metadata only really makes sense to access by searching for a key anyway.

dangermattic · 2025-09-18T10:00:46Z

	1 Warning
⚠️	This PR is assigned to the milestone 23.3. This milestone is due in less than 2 days. Please make sure to get it merged by then or assign it to a milestone with a later deadline.

Generated by 🚫 Danger

Decode meta_data incorrectly sent as an object

060141d

joshheald added this to the 23.3 milestone Sep 17, 2025

joshheald added the type: task An internally driven task. label Sep 17, 2025

joshheald added 2 commits September 17, 2025 18:40

Use a simpler metadata decoding approach

c574960

Remove whitespace

5746bc6

joshheald requested a review from staskus September 17, 2025 17:48

joshheald modified the milestones: 23.3, 23.4 Sep 17, 2025

joshheald added 3 commits September 17, 2025 19:51

Ensure subscription details are decoded

c66e524

Fix subscription parsing

2d989a2

Simplify the decoding

fd8e12e

joshheald marked this pull request as ready for review September 17, 2025 19:46

joshheald added 5 commits September 18, 2025 05:12

Remove unused helper functions

7696d51

Simplify tests

d26c2be

Revert unnecessary formatting changes

177b484

Remove unnecessary import

fe6f10c

Add meta_data workaround to 23.4 release notes

465ed5c

staskus requested a review from Copilot September 18, 2025 06:40

staskus self-assigned this Sep 18, 2025

Copilot AI reviewed Sep 18, 2025

View reviewed changes

staskus approved these changes Sep 18, 2025

View reviewed changes

correct test name

3d57295

Co-authored-by: Povilas Staskus <[email protected]>

joshheald added 4 commits September 18, 2025 10:42

Move workaround to 23.3 release

ae1c3df

Remove unused codable conformance

501b660

Remove unnecessary internal specifiers

afe445f

Log when we fall back to an empty array

fa134fa

joshheald enabled auto-merge September 18, 2025 10:00

joshheald modified the milestones: 23.4, 23.3 Sep 18, 2025

joshheald merged commit 3fd0ab9 into trunk Sep 18, 2025
15 of 16 checks passed

joshheald deleted the woomob-1362-decode-productvariation-meta_data-provided-as-an-object-not branch September 18, 2025 10:20

-            return Array(metaDataDict.values)
+            return metaDataDict.sorted { (lhs, rhs) in
+                (Int(lhs.key) ?? 0) < (Int(rhs.key) ?? 0)
+            }.map { $0.value }

Decode meta_data incorrectly sent as an object #16141

Decode meta_data incorrectly sent as an object #16141

Uh oh!

Conversation

joshheald commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Steps to reproduce

Test information

Uh oh!

wpmobilebot commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joshheald commented Sep 17, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

staskus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshheald commented Sep 18, 2025

Uh oh!

dangermattic commented Sep 18, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Decode `meta_data` incorrectly sent as an object #16141

Decode `meta_data` incorrectly sent as an object #16141

joshheald commented Sep 17, 2025 •

edited

Loading

wpmobilebot commented Sep 17, 2025 •

edited

Loading